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ABSTRACT 



Getting real educational benefits from investments in 
technology requires careful planning. Based on the experience of implementing 
PLATO [R] at over 5,000 sites, a nine-step planning model for implementing 
technology was developed. The nine planning tasks are based on the 
understanding of the factors that are critical to the success of most 
educational applications of technology, including PLATO [R] . The steps are: 

(1) Get buy-in from key personnel; (2) Decide on program goals; (3) Decide on 
instructional applications of technology; (4) Develop instructional models 
for the applications; (5) Develop an instructional management plan; (6) Plan 
hardware/ software deployment; (7) Plan technical support; (8) Plan 
professional development; (9) Plan evaluation. For each step, this Technical 
Paper describes the key concepts and principles involved, and provides 
suggestions to help with planning. (AEF) 
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Executive Summary 



Getting real educational benefits from your investment in technology 
requires careful planning. Based on the experience of implementing 
PLATO at over 5,000 sites, we have developed an 9-step planning model 
for implementing technology. The steps are: 

1. Get buy-in from key personnel. 

2. Decide on program goals. 

3. Decide on instructional applications of technology. 

4. Develop instructional models for the applications. 

5. Develop an instructional management plan. 

6. Plan hardware/software deployment. 

7. Plan technical support. 

8. Plan professional development. 

9. Plan evaluation. 

For each step, this paper describes the key concepts and principles 
involved, and provides suggestions to help you plan. 
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Introduction 



In deploying educational technology, as in life, there are no guarantees of a 
positive outcome. Hardware can easily become “shelfware.” Software can 
be forgotten in the backs of countless desk drawers. Without careful 
planning and implementation, it’s likely that the educational return on the 
technology investment will be very low. There is a generation of research 
demonstrating the potential of computers to improve learning, but there's 
much less research on what it really takes to realize those benefits. 

Our experience in implementing PLATO systems at over 5,500 sites has 
allowed us to work first hand with educational organizations who have 
gotten real, meaningful, measurable results from technology - and those 
who have not. We have learned that even the best instructional software 
can be no better than the way in which it is used. Teachers can use 
technology as an effective lever for meaningful change in their role in the 
classroom — or the technology can merely act as an expensive alternative to 
workbooks, overhead projectors, ditto sheets, encyclopedias and board 
games. Technology can support and strengthen the very core of the 
teaching and learning experience, or it can be relegated to a role which is 
peripheral at best, and a distraction at worst. The challenge is to develop 
a technology implementation plan when introducing technology, to make 
sure that all the critical success factors are in place, when they are needed, 
to achieve real benefits. 

In this technical paper, we'll summarize some of the major “lessons 
learned” as we have worked with our clients. Our discussion will focus 
mostly on curriculum, instruction, and instructional management, rather 
than on the “bits and bytes” of the hardware and software. Our discussion 
will center on PLATO, of course, but PLATO is an open system which can 




Copyright ©TRO Learning, Inc. January 2000 All rights reserved. 
Tech Paper #5-Guide to Implementing Technology.doc 



5 



Page 3 



serve as the “backbone” to help you integrate a wide range of software 
resources, so we’ll talk about the full range of alternatives for use of 
technology and how they all fit together. 1 

To get the most from your technology investment, there’s a lot to do. Begin 
implementation planning as early as possible; ideally, before the final 
decisions have been made on what hardware and software to buy, and how 
‘ much to budget for professional development. But the good news is that 
our experience has shown that you can start getting some benefits almost 
immediately, with a simple implementation plan. As you and your 
colleagues become familiar with PLATO, you can try more sophisticated 
approaches. Over time - often two or three years, in our experience - the 
greatest benefits will emerge. As your sophistication grows, PLATO’s 
Education Consultants will be your primary resource for workshops, in- 
class consultations, and planning. Supporting them is a network of senior 
education and technical professionals, who can help you achieve your most 
sophisticated goals for improving learning with technology. 

The implementation planning process described in this paper has these 
steps: 

1. Get buy-in from key personnel. 

2. Decide on program goals. 

3. Decide on instructional applications of technology. 

4. Develop instructional models for the applications. 

5. Develop an instructional management plan. 

6. Plan hardware/software deployment. 

7. Plan technical support. 

8. Plan professional development. 

9. Plan evaluation. 

For each step, we will describe the key concepts and principles involved, 
then provide suggestions to help you plan. 



1 A useful discussion of the full range of educational software types is in Technical 
Paper # 6 . 
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Step 1 : Get Buy-In from Key 
Personnel 




The first step in planning any change is to identify the key personnel who 
will be responsible for implementing the change, and to get them 
committed to the project. Introducing technology to a school or district is 
no different from any other major change in this respect. In our 
experience, the key roles at the building level are the Instructional Leader 
and the Technology Manager . Backing up these people at the district level 
are the technical support specialist, key administrator(s), and (for PLATO) 
the Education Consultant. All of these people must help develop the goals 
of the technology implementation (see Step 2, below), and must be 
committed to achieving those goals. 

Here's a summary of each role: 

Instructional Leader: Our experience parallels the research on innovation 
in education: for the innovation to succeed, there must be a strong 
building-level instructional advocate. This person is often the Principal, 
but can also be a department or program chairperson, or even a master 
teacher or media specialist. The person must be respected as an “opinion 
leader” among the teaching professionals in the school who will be using 
the technology. He or she will play the key role of persuading the teachers 
to commit to achieving the goal(s) (see Step 2, below), and will help the 
teachers incorporate the technology into their teaching in the intended 
way. In any technology implementation, this is the most key role. If there 
is no strong instructional leader at the building level, the odds are 
overwhelming that the technology innovation (or any other major 
instructional innovation) will fail. If the person leaves the project at any 
point, he or she must be replaced as soon as possible. 

Early Adopter: This person may be the same as the Instructional Leader, 
but often is someone else selected by the Instructional Leader. This 
person often “leads by example,” and is the “early adopter:” the first to 
successfully incorporate the technology into his or her teaching by 
demonstrating that it works, and by demonstrating that the risks of 
adopting the technology are minimal. 

Technology Manager . In addition, each building needs to have at least one 
teacher, media specialist or lab manager who is trained in the day-to-day 
system maintenance and administrative tasks. This is the person who can 
help set up on-line curricula and learner records, perform regular backups, 
clear viruses and maintain system security, help recover from the 
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inevitable computer glitches, and help run administrative usage reports. 
Since classwork often stops unless these tasks are performed immediately, 
this person must be available and “on call” for immediate response 
whenever school is in session. Because availability is so crucial, it’s wise 
to have two or more people trained to perform these tasks. 

Many schools prefer to combine this technology management role with a 
teaching role, since this person has the most daily contact with teachers 
and learners as they master the technology. This is often desirable, but in 
our experience the minimum technical part of the role (as opposed to the 
teaching part) can be filled by an uncertified teacher aid, or even an 
extraordinarily dedicated parent volunteer. PLATO training for this 
person is required. 

Technical Support. Our experience also shows that effective and 
responsive technical support is critical to any successful implementation. 
No technology innovation can succeed unless the system is available to 
learners reliably each day. Unfortunately, the technology of networks, the 
Internet, Windows, and so on, requires periodic attention by trained 
technicians. For any school or district with a few hundred (or a few 
thousand) computers, maintaining the technology is likely to be a full-time 
job for trained and certified computer professionals. For smaller 
installations, a service contract with a local network specialist who is 
certified in the network software you are using (Windows NT Server or 
Novell) may suffice. Technical services are also provided by PLATO’s 
Field Engineers. 

Administrators. Timely allocation of adequate resources for teacher 
training, ongoing support, instructional software, and hardware is critical 
to the success of any technology implementation. In addition, key 
administrators must make the success of the technology implementation a 
priority for all those involved. Administrators must monitor the 
implementation of the technology. Finally, they must make sure that 
teachers, and all those involved, receive feedback on their performance in 
achieving the goals of the technology implementation. 

The relevant administrators may be at the building or district level — or 
both. The one(s) to involve in the project are the one(s) who perform the 
tasks described above. A common practice among school districts is to 
administratively separate technology from the day-to-day administration 
of the district, and from curriculum coordination. This can create the 
impression among “line” administrators and curriculum coordinators that 
they “don’t have to do technology.” It’s important for all parties take 
special precautions to keep the technology function fully involved in the 
goals of the district. 

Education Consultant. The PLATO Education Consultant (EC) has special 
training in the use of PLATO in the classroom. It’s important to involve 
the EC as early as possible in the planning of the technology 
implementation — often, before the contract is signed, but in any case 
before the hardware and software are delivered. The EC’s role is to help 




Copyright ©TRO Learning, Inc. January 2000 All rights reserved. 
Tech Paper #5-Guide to Implementing Technology.doc 



Page 6 



you think through the implementation plan, to assist as needed with 
installation of the software as needed, to conduct appropriate teacher 
training workshops, and to provide ongoing follow-up consultation with all 
teachers who are using PLATO. 

Additional senior specialists are available to back up the PLATO EC. 

These specialists include both educators and technical specialists in 
networking and software. They are available to work with the EC and 
with you to make your technology implementation a success. 

For non-PLATO software, it may be possible to fill the role with a 
consultant who serves the entire district or regional service organization. 
The importance of this role has been well documented in the change 
management literature, and is analogous to the “county agricultural 
agent.” This person should be primarily trained as an educator, together 
with training in use of technology in classrooms, and technical expertise as 
needed. 
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Step 2: Decide on Program 
Goals 



It's important to decide first what you want the technology for. This 
decision should guide all your decisions about what software and hardware 
to buy and how to use it. Unfortunately, many schools and districts work 
the other way: first they let “experts” decide what hardware to buy, with 
minimal input from instructional staff. When the hardware arrives, the 
teachers face the task of deciding what to do with it, often with little 
leadership. 

It’s important for the key personnel you identified in Step 1 to be 
committed to the program goals. This is usually best done by involving the 
program’s key personnel in developing the goals, and committing the 
resources necessary to achieve them. 

Sound goals deal with meaningful changes in learning outcomes and/or 
instructional processes. Unsound goals often deal with technology or 
simple replacement or augmentation of existing instructional practices. 
Here are some examples of sound and unsound program goals: 

* Improve scores on our 10 th grade state competency test.” 

This is an example of a sound goal, because it deals with a measurable 
learning outcome (scores on the competency test), and states a direction of 
change (improvement). 



“Put 5 computers in every classroom , with Internet access through a wide 
band network.” 
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This is an example of an unsound goal, because it deals with deployment of 
the technology without regard to any expected changes in teaching and 
learning. 



“Strengthen teaching of NCTM standards related to authentic math problem 
solving” 

This is an example of a sound goal, because it deals with instructional 
processes and learning outcomes. 



“Use the Internet in teaching Science.” 

This is an example of an unsound goal, because it says nothing about a 
teaching process or a learning outcome, only that a technology (the 
Internet) should be used in a classroom (Science). An example of a sound 
goal might be, “Use authentic data (obtained via the Internet) to provide a 
context for analysis and interpretation of local environmental 
measurements.” 



“Use the Computer to do Projects.” 

This is an example of an unsound goal, because it does not include a 
learning outcome - only an output (the project). A learner who builds a 
PowerPoint file or a Web site doesn't necessarily learn any more about 
science (or social studies, or literature, etc.) than one who hand-writes a 
paper — though he or she may show more motivation, and may learn 
something about teamwork and using the computer tools (which might be 
acceptable secondary goals). A better goal might be, “ Construct a project 
which compares at least 3 conflicting positions on an issue , by summarizing 
and juxtaposing the competing points and providing hyperlinks to 
supporting arguments and data.” 



“Ensure that all entering freshmen are ready to do grade-level work in math 
and language arts” 

This is an example of a sound goal, because it describes a learning outcome, 
implies measurement of the outcome, and implies an instructional process 
(remediation). 



“Enable at-risk students (whether due to mobility / drop out, pregnancy, 
adjudication, illness, etc.) to earn lost credits and graduate no more than 1 
year late.” 

This is an example of a sound goal, because it describes a learning outcome 
and implies measurement of that outcome. 




Copyright ©TRO Learning, Inc. January 2000 All rights reserved. 
Tech Paper #5-Guide to Implementing Technology.doc 



11 



Page 9 



“Devote 50% of my class time to problem-based activities, by moving 50% of 
expository teaching to self -instructional formats used in or out of class.” 

This is an example of a sound goal, because it describes a teaching process 
change goal — even though it does not describe a learning outcome (the goal 
would be even stronger if it did this also). 



“Every learner will use the computer for at least 30 minutes per week.” 

This is an example of an unsound goal, because it deals only with 
deployment of technology, and is not concerned with learning outcomes or 
teaching processes. 



Notice that many sound goals don’t mention technology at all — and that if 
technology is mentioned, it’s only as a means to a learning outcome. That’s 
appropriate because the mission of schools is learning, not technology 
deployment. It sounds obvious, but this basic principle is often overlooked 
in practice. 

A single large-scale technology investment may have many sound goals, of 
course. Different classes can use the same hardware for different goals. It 
may even be reasonable for the hardware to be used during the day by the 
school, in the evening by an adult program, and on the weekends by a 
community-based program - there’s no reason to ever turn off the 
computers, and such uses could even help cost-justify or pay for the 
equipment 2 . The point is to make sure all of the technology users have 
clear, sound and reasonable (feasible) goals for changing learning outcomes 
and/or teaching/learning processes. 



2 



Some of our more entrepreneurial clients purchase PLATO and charge back its off- 
hours use by hourly or monthly charges to local adult learning, job training and 
community-based organizations which run weekend homework/tutoring programs. 



O 
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Step 3: Decide on the 
Instructional Applications of 
Technology 



There are many different types of software, and they are designed to serve 
many different purposes in an instructional program. Clever teachers can 
adapt an even broader array of software to instructional uses, even when 
that was not part of the software’s original intent. But all classroom 
applications of software are not created equal: different types foster 
different uses, and have different learning outcomes. It’s therefore 
important to think carefully about what kinds of software are best suited 
to the goals you’ve identified. As you do so, you’ll also be thinking about 
the instructional models you’ll want to use, and even the instructional 
management system you’ll need. 

In the next section, we’ll provide a brief overview of kinds of software with 
educational use. A more detailed dicussion is in Technical Paper #6. 



Instructional Software Types 

It may be useful to think of three general types of educational software: 

• Supplementary'. Software which adds little or no new content, and 

parallels teaching already done in other modes. Electronic alternatives 
to textbooks, lectures, workbooks, etc. Examples include skill and 
memory games and exercises; on-line references and informational Web 
sites, and many on-line tests. PLATO’s Vocabulary Builder, practice 
lessons and on-line tests fit here. 
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• Complementary: Software which adds new content to the curriculum, 
often in ways for which there is no non-computer alternative. 

Examples include simulations (such as PLATO Problem-Solving 
Activities) and games designed to teach problem-solving; Web-based 
“expeditions” and 4 projects.” Tools which automate low-level tasks 
(such as word processors, data graphing tools, web page authoring 
systems, search engines, etc.) also fit, if they are used as part of a 
learning activity which adds some new topic to the curriculum; by 
themselves, the tools (and most Web sites) have little or no support for 
specific learning activities other than learning to use the tool itself. 

• Primary: Software which acts as the main source of initial teaching, as 
a replacement for non-electronic modes of instruction. The purpose is 
to allow the teacher to work in a “guide on the side” mode, instead of 
“sage on the stage.” PLATO tutorial lessons are examples. 

A given software product can often be used in more than one of the three 
ways, so notice that these definitions characterize the way in which 
software is used as much as how the software is designed. 

There are a number of tradeoffs associated with the three uses. The 
tradeoffs concern learning gains, teacher training, ease of implementation, 
and hardware deployment: 

• Our experience is generally consistent with the research finding that 
supplementary uses have the least potential for producing large 
learning gains, though incremental improvements are possible. 
Complementary and primary uses have much greater potential, if they 
are applied broadly across an entire curriclum or program. 

• Teachers seem to find that the supplementary uses are the easiest to 
implement and require the least training, in comparison to the other 
two types of uses. 

• Supplementary and complementary uses can be added to existing 
classroom routines and ways of teaching fairly easily. Primary uses for 
a whole class are often difficult to implement within a conventional 
school structure, and are much more common in special programs, 
alternative and charter schools, etc. 

• Supplementary and complementary uses of many kinds (but not all) can 
be implemented using a student:computer ratio of 4:1 or so. The 
common recommendation to place 4-6 computers in every classroom 
seems to be based on an unstated assumption of supplementary or 
complementary use. By contrast, most (but not all) primary uses 
require a student:computer ratio of 1:1 or 1:2 (for the portion of the 
classes being taught this way). This leads to decisions to give each 
learner a laptop computer, or to place 12-30 computers in a classroom, 
or to set up computer labs. Thus, deciding on what type of software 
uses you want can give you a rough estimate of the number of 
computers you will need. 
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Step 4: Develop Instructional 
Models for Applications 



The next step in your planning is to work with your instructors to plan in 
greater detail how the technology will be integrated into the curriculum. 
To do so, you need to answer these questions: 

1. What is the learning goal of the technology application? 

You answered this question in Step 2 (above). However, at this stage you 
should have additional details of exactly what parts of the curriculum will 
be taught to which learners using technology, based on your analysis of 
general goals in Step 2, and the kinds of software you want, based on your 
analysis of software types and resource requirements in Step 3 (above). 

2. How will we assign learners to use the technology? 

Some instructional models assume everyone will be doing the same thing 
at the same time. Others require sophisticated individualized learning 
plans (ILP’s) based on an assessment of individual needs. In these cases, 
placement of the each learner in the right assignment, on a daily basis, 
should be an important factor in your planning. 

3. What will the learner's role be? 

Instructional models vary widely in how much and what kind of decisions 
the learners make about their own learning. Some models require solo, 
self-paced work, while others require collaborative study with everyone 
studying the same thing at the same time. It's important to work out in 
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detail what the learner’s responsibilities will be and how they will be 
fulfilled. 



4. What will the instructor’s role and program structure be? 

Once the learner’s role is defined, then it’s necessary to plan the role for 
the instructor, and the overall program structure, so that both will lead 
the learners to fulfill their roles. 

5. How will we manage the resources? 

In Step 3 (above), you worked out a rough estimate of resources required. 
With the decisions made about curriculum and instruction, it’s possible to 
develop a detailed plan for how to schedule and manage the hardware and 
software resources. 

6. How will we assess the learners? 

You also need to decide how you will assess what the learners have 
learned as they have used the technology. For example, PLATO has a 
range of powerful assessment options built in. Other software leaves 
assessment as a task entirely for the instructor. 



Technical Paper # 6 presents four “generic” instructional models for 
PLATO. The four models are: 

Review/Reinforcement (Supplementary) 

The goal of the review/reinforcement instructional model is to reinforce 
the knowledge and skills of the learner using any PLATO curriculum. 

Primary instruction is assumed to be done in the classroom, without use of 
PLATO. PLATO lessons are assigned (often as seat work or out-of-class 
work) before a given classroom lesson to review prerequisite concepts, or 
after a classroom lesson to provide additional reinforcement, review and 
practice of the topics taught in class. The PLATO work can occur 
immediately following the classroom instruction, or after a delay (such as 
for end-of-unit review or review before a unit test, final exam, or 
competency test). 

Enrichment (Complementary) 

The goal of the enrichment instructional model is to add to or deepen the 
knowledge and skills of the learner through use of any relevant PLATO 
curriculum. 

Primary instruction is assumed to be done in the classroom, without use of 
PLATO. PLATO lessons and other on- and off-line materials are assigned 
(often as seat work or out-of-class work) by the instructor or selected by 
the learner. Study is usually after a classroom lesson, and is used to 
provide additional knowledge and skills for learners who want more 
depth, background, or advanced work in a subject. The computer work can 
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occur immediately following the relevant classroom instruction, or in 
support of an independent research or learning project. 

Problem-Centered (Complementary) 

The goal of this instructional model is to make problem-solving the central 
strand of the curriculum. Development of knowledge and skills is done in 
the context of problem-solving, as a pre- or co-requisite. The intended 
results include deeper understanding, greater transfer to non-school tasks, 
development of learning skills, and greater motivation. 

At the center of each unit is a problem-solving activity which can be a 
PLATO Problem Solving Activity (PSA), or other case problems 
implemented on the computer or offline. Knowledge and skills 
development is done in the context of the problems, as a pre- or co- 
requisite. This establishes a context for the learning of facts, concepts and 
skills. 

Skill Development System (Primary) 

The goal of this instructional model is to develop, remediate, and/or 
enhance the knowledge and skills of the learner using PLATO. 

This is a competency-based, mastery instructional model. It is completely 
individualized, and is based on three principles: (1) learners should study a 
topic only when they have fully mastered its prerequisites; (2) learners 
should study a given topic until they have fully mastered it, before going 
on; (3) learners should study only topics they have not already mastered 
(further discussion of these principles is in the next section of this paper). 

This is a particularly success-oriented model. In principle, anyone who is 
ready to learn can work until they have mastered each skill, in privacy. 
There is no comparison with peers — only with progress toward personal 
goals. In principle, in this model failure is impossible — only non- 
completion. 



For each of the four models, Technical Paper #6 answers the six questions 
above. Of course, there are many different instructional models using 
many different types of software, and not all models are equally effective 
in meeting your needs. The four models summarized above are intended to 
serve as a starting point for you to develop instructional models which 
meet your needs. 
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Step 5: Develop an Instructional 
Management Plan 



Instructional management may be an unfamiliar term, but it's not an 
unfamiliar set of activities. Each day, instructors and administrators 
decide 

• who is going to go to which classroom and instructor 

• what learning activities will occur 

• which texts and resources they will use 

• how the resources will be scheduled, how long each activity will be 

• who will be assessed and how 

• how the assessment will guide future work 

• how all these decisions will be communicated to learners, parents, 
staff, and others who need to know 

One of the big advantages of technology is the ability to do instructional 
management much more precisely and flexibly than is possible using 
conventional methods. Using conventional methods, many of these 
decisions are dictated by century-old custom and by what's feasible in a 
paper-and-pencil world. For example, the convention of the 25-person 
class in secondary schools is probably due more to practical limitations of 
paper-and-pencil instructional management than to any research on class 
size. Even fixed class schedules, “one size fits all” curricula, and practices 
such as relatively infrequent and large tests (rather than frequent small 
ones) are probably artifacts of a paper-and-pencil instructional 
management system. In that system, highly individualized instruction 
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almost always creates an unacceptable work load for instructors, and leads 
to “burn-out” within a few years. 

Used appropriately, technology can change these things. Large-scale and 
fine-grained individualization becomes possible, with precise alignment to 
curriculum standards and frequent (daily, or even hourly) progress 
assessment with immediate feedback. “Just in time” delivery of individual 
instructional prescriptions, any time and anywhere, can be routine. 
Learners really can work at their own rate, and follow their own interests 
and needs, without concern for appearing to be a “nerd” or a “dummy.” 

The critical social context for learning can be based on success in a goal- 
oriented community, rather than being based on individual competition 
and fear of failure. And, these advantages can be obtained while freeing 
instructor time to work as a “guide on the side” with each learner (one 
instructor using PLATO in this way recently commented that for the first 
time in 25 years of teaching, he felt he could really teach; another 
commented that for the first time she thought of her learners as 
individuals, rather than a class). 

PLATO’s instructional management capabilities are built into the PLATO 
Pathways computer-managed instructional system, and are founded on the 
basic modular and open architecture of the system. You can use Pathways 
to prescribe and track the day-to-day work by your learners, whether or 
not PLATO courseware is involved, and whether your system is based on a 
local area network, the Internet, or even a single work station. 



Planning An Instructional Management System with 
PLATO 



To plan your instructional management system, you will have to make a 
number of practical decisions which need to be consistent with the type(s) 
of software you have chosen and the instructional model(s) for their use. 
The following table outlines some of the most common decisions you will 
need to make, and briefly mentions some of the options available to you 
through PLATO. 



Decision 


Discussion 


PLATO Options 


Which 
learners 
will use the 
computers 
? 


A common error is to try to spread 
computer use evenly across all 
learners in all classes. This usually 
results in so little computer time per 
learner that little or no improved 
learning results. 

Better is to choose learner groups 
based on the goal(s) you decided upon 
in Step 1. 


If you use PLATO on your 
Local Area Network 
(LAN), or on the Internet, 
you can make it available 
throughout your school, 
district, and even in the 
learners’ homes and in 
the community. 


How 


A common error is to give all learners 


PLATO’s centralized 
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Decision 


Discussion 


PLATO Options 


should we 
schedule 
computer 
use? 


an equal amount of time on the 
computer, even if this only works out 
to 20 minutes a week or so. This 
practice virtually precludes significant 
improvement of learning for any 
learners. 

It makes more sense to plan for at 
least 1 hour daily, at least 2 days per 
week, per subject (3-5 days/week 
preferred), at a learner:computer ratio 
determined by the type of software you 
plan to use. 

Depending on your program goals, this 
level of access may not be needed for 
the full year, so different classes may 
be able to share computer facilities 
without undue interruption of their 
curriculum sequence. To reach this 
level of access, you may need to 
abandon some program goals and focus 
computer use on certain high-need 
learners or high-priority uses. 

If you are using mastery learning (the 
skill development model), then plan to 
accommodate a 6:1 ratio of completion 
times (see Appedix A, “Mastery 
Learning and PLATO”). You may be 
able to do this by having slower 
learners continue work out of class, 
before or after school, or at home via 
the Internet. 

There is no reason ever to turn off the 
computers. Make them available as 
continuously as possible. 


management and 
prescription over LAN 
and Internet systems 
gives you maximum 
flexibility in scheduling. 

Since PLATO will run via 
the Internet on many 
home computers, you may 
be able to supplement 
computer time at school 
with computer use at 
home. 


What 
should the 
instructors 
do while 
the 

learners 


A common error is to assume that 
when the learners are on the 
computers, there is nothing for the 
instructors to do and they can be 
assigned to other tasks. 

It’s important for instructors to be 


Research on computer 
learning shows that 
learning approximately 
doubles when an 
instructor is effectively 
involved. PLATO is 
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Decision 


Discussion 


PLATO Options 


are on the 

computers 

? 


actively involved in a “guide on the 
side” role. Instructor activities vary 
depending on the goal, the type of 
software used, and the instructional 
model in use. Training usually is 
necessary to acquire the skills for this 
role. 

This involvement does not always have 
to be in “real time,” however, 
especially with more mature or self- 
directed learners. In distance 
education settings, instructor 
involvement can be done through 
asynchronous communications such as 
e-mail and chat groups. 3 Note that it is 
frequently reported that the instructor 
workload in distance education 
courses is about double that of 
comparable classroom teaching. 


designed for solo use, but 
our evaluation data are 
consistent with the 
general research: active 
involvement of an 
instructor in the “guide 
on the side” role greatly 
improves learning, across 
all instructional models. 

PLATO Pathways 
provides a powerful 
system to provide usage 
and progress reports to 
instructors on learner 
progress in class, in 
specific groups, or even 
by exception to “flag” 
those who need extra 
attention. 

PLATO includes e-mail in 
LAN and Internet 
systems, and chat groups 
are available as well on 
the Internet. 


How 

should we 
group the 
learners? 


Learning is a social experience. It’s 
important to create a positive, success- 
oriented experience for all learners. 
For learners with poor teamwork 
skills or a history of negative group 
interactions, solo work on the 
computer can bypass this problem and 
provide a positive learning experience. 
At the same time, software types 
which support collaborative learning 
and peer teaching can help build 
teamwork and interpersonal 
communication skills. 

It’s also important not to group 
learners so as to create the impression 


PLATO’s mix of solo 
tutorials and 
collaborative learning 
software can support a 
mix of peer teaching and 
small-group 
collaboration. 

PLATO Pathways also 
allows you to integrate 
non-PLATO activities 
such as tools and web 
sites which are useful for 
inter-group 
communication. 




3 For further research on distance education and PLATO, refer to Perez, S. and Foshay, 
R., Adding Up the Distance: Critical Success Factors for Internet-Based Learning in 
Developmental Mathematics. Monograph and multimedia CD-ROM published by the 
League for Innovation in the Community College, March, 2000. 
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Decision 


Discussion 


PLATO Options 




that the computers are for the “geeks” 
or the “dummies.” If you use the 
individualization capabilities of your 
software, you can intermix groups of 
varying abilities without creating 
problems with learning rates. Group 
by common learning goals. Encourage 
groups to communicate among 
themselves and with the instructor(s), 
and even with peers in other locations 
or cultures. 

Of course, testing should be done in a 
solo, secure environment. 


PLATO Pathways allows 
you to monitor individual 
progress when learners 
are in groups, as long as 
testing is in a solo, secure 
environment. 

Pathways also allows you 
to import and export data 
from and to your school 
administrative software. 
This can include not only 
learner names, but 
important demographic 
descriptors which may be 
useful for grouping and 
tracking performance. 


How will 
we 

integrate 
computers 
into our 
curriculum 

? 


A common error is to underestimate 
the degree of planning needed to 
effectively integrate technology. 

Steps 2 through 4, discussed above, 
will help you make the decisions you 
need to effectively integrate 
technology. Simply having learners 
“working on the computer” will not 
lead to meaningful gains. 

Be particularly careful to maintain 
close and precise alignment of all 
learning experiences, both on- and off- 
line. For example, it can be profoundly 
disorienting to learners to rapidly 
switch from study of one math topic in 
a large-group class, to another math 
topic on the computer. Concern over 
this issue is often a major reason why 
instructors don’t individualize. 

It’s also important to see that all of 
your curriculum standards are taught 
by appropriate learning experiences 
(whether on- or off-line), and that they 
are in a logical sequence with no 
“gaps.” 


PLATO’s modular 
structure, large coherent 
curricula, and custom 
learning path capabilities 
in Pathways are designed 
specifically to support 
curriculum alignment to 
your state and local 
standards. 

PLATO’s flexible 
assessment options may 
allow you to perform 
many of your assessment 
requirements on-line, on- 
demand and 
automatically without 
instructor paperwork. 

You can use Pathways to 
add in non-PLATO 
computer and non- 
computer learning 
activities, and track 
usage and results. When 
you do this, Pathways 
becomes the primary 
classroom instructional 
management tool. 


How will 


In any individualized system, it’s 


PLATO uses module 
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Decision 


Discussion 


PLATO Options 


we track 
and assess 
learner 
performan 
ce? 


important to keep a close watch on 
learner progress, so you can intervene 
immediately when a learner 
encounters a problem. Choose a 
computer system which can assess 
learner progress frequently (daily or 
even more often), and which can report 
that information to the you in a 
convenient form which makes it 
possible to identify “problem learners:” 
those who are floundering or not 
trying. 

Also: make sure the tests are tied 
closely to the curriculum and to your 
curriculum standards, so the 
information on learner progress will be 
meaningful. 

For more complex and higher-level 
learning objectives, you may wish to 
use a portfolio assessment approach. 
Make sure your software can generate 
the information you need for this 
purpose. Many PLATO Problem 
Solving Activities (PSA's) can do so. 


mastery tests (small 
quizzes which accompany 
every 45-minute module) 
and cumulative course- 
level tests, which can be 
assembled on a custom 
basis. You can even add 
your own tests if needed. 

The tests communicate 
with PLATO Pathways, 
the management system, 
to provide real-time 
progress data in 
convenient and efficient 
formats. 


How will 
we report 
learner 
progress? 


Instructors are used to end-of-unit and 
end-of-semester grades. The computer 
can provide specific details on 
progress for individual learners, and 
do it daily. This can be very 
motivating for some kinds of learners, 
as well as allowing instructors and 
parents to intervene quickly when a 
problem occurs. 

In the “Skill Development Model,” it's 
feasible for the computer to use skill 
profiles rather than just cumulative % 
scores or letter grades. This can be 
much more diagnostic, and it can 
recognize the different kinds of 
intelligence your learners have. 


PLATO Pathways can 
generate skill profiles 
based on your 
curriculum's design, and 
report on daily or weekly 
progress. 
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Step 6: Plan Hardware/Software 
Deployment 



Closely related to the instructional management plan is your plan for 
deploying the technology. A common error is to make decisions about 
hardware, software and support before the goals and instructional plans 
for use of the technology are decided. In this section, we'll discuss some of 
the instructional considerations which should affect your technology plans. 
Details of the technologies themselves are beyond the scope of this 
discussion. 

How many computers and software licenses do we 
need? 

You can use the taxonomy of software types presented in Technical Paper 
#6 to decide on: 

• the type(s) and quantity of software and hardware you need 

• the student:computer ratio you must provide 

• the professional development you must plan and budget for 

We'll discuss the first two decisions here, and the last one under Step 8. 
Some Program-Based “Rules of Thumb” For Budgeting 

Use this procedure as a general guide to make your decisions about the 
type of software, the range of topics to be covered, and the number of 
computers you will need to attain your learning goals: 



Step 


Example 


1) For each goal, decide if it requires 
a supplementary, complementary or 
primary strategy (refer to Parts 2 and 
3, above). 


A High School decides to improve pass 
rate on its state competency test of 
mathematics. The school decides on a 
primary strategy. 


2) For each strategy, decide on the 
software type(s) which you think 
should be the main one(s) for that 
goal. 


The school decides to use a combination 
of tutorial and problem-centered 
software. 


3) For each software type, look up 


Tutorial software: 1:1 
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the suggestion for ideal 
learner:computer ratio. 


Problem -centered software: 4:1 


4) Divide your total student 
population using that software type 
by the numerator of the 
student:computer ratio. 


400 learners who failed the test in the 
first year will be remediated. 

400/1=400 computer work sessions 
needed in each daily “cycle”. 


5) Multiply the product of (4) by the 
number of computer work sessions 
per week each learner is to have. 

• For primary applications, assume 
2-5 work sessions per week (3 is 
often optimal). 

• For complementary applications, 
assume 2-5 work sessions per 
week (usage patterns vary widely 
depending on the project) 

• For supplementary applications, 
assume 2-3 work sessions per 
week (usage patterns vary widely 
depending on scope of the 
curriculum) 


3 x 400 = 1,200 computer work sessions 
needed per week. 


6) Calculate the total number of class 
periods (including after school time, 
lunch time, etc.) during which the 
computers are to be available for 
study on this topic. 


9 50-minute periods in a day, counting 
lunch time and 1 after school session. 5 
days per week. 

9 x 5 = 45 periods of use (minimum 
available) 


7) Divide the product of step (5) by 
step (6). 


1200 / 45 = 27 student work stations 
needed (assuming 100% scheduling 
efficiency). 


8) Decide what fraction of 100% the 
computers will actually be 
scheduled for use by learners 
working on this goal (no 
scheduling system is 100% 
efficient, so the computers will be 
idle some times). Reduce the 
fraction further if self-pacing is 
planned, to allow for the 6: 1 ratio 
of completion times. Multiply the 
product of step (7) by this factor. 

• In computer labs, scheduling 
efficiency can be very high 


Assume 80% scheduling efficiency, to 
allow for scheduling flexibility and 
additional use by learners who need it. 

Multiply number of student-periods 
needed by 1.25. 

27 x 1.25 = 34 student work stations 
needed. 
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(perhaps 80%). 

• In classrooms, scheduling 

efficiency rarely exceeds 30% or 
so. 

For a discussion of tradeoffs in 
deploying computers, see Step 9, 
below. 




9) Add work stations for 


3 computers added to faculty offices, 5 


administrative and teacher lesson 


added for unscheduled use and 


planning, and to allow for 


maintenance. 


maintenance, and unscheduled use, as 
needed. 


Total work stations needed = 34 + 8 = 42 


Maximum software licenses needed=40. 



In our experience, the hardware/software requirements you calculate by 
this procedure often exceed “one size fits air recommendations such as 5 
computers per classroom. This procedure provides a sound basis for 
justifying a hardware and software budget based on program goals. If the 
decisions have already been made, then it's important to set priorities 
among the goals, so that you will be held accountable only for achieving the 
ones for which resources are available. 

Where should we put the computers? 

A common error is to put a few computers in each classroom without 
considering how they will be used. There are tradeoffs in both directions 
for putting computers in classrooms, centralizing them in labs, or giving 
each learner a personal laptop computer. If the budget allows, you may be 
able to do more than one kind of deployment. 

In addition, remember that many of your learners may already have 
computers at home. Through the Internet, these learners may be able to 
use some kinds of software, such as PLATO. A number of innovative 
programs have pioneered easy financing of home computers and laptop 
computers for educational use by families and by instructors. This kind of 
access can greatly increase the options you have for scheduling flexibility. 



4-6 Computers/Classroom 


25+ Computers/Lab or 




Personal laptops for each learner 


Advantages 


Advantages 


• Supports small-group instructional 


• Supports 1:1 instructional models as 


models 


well as small-group models 


• Allows spontaneous use and 


• Useful for planned professional 


integration into other activities 


development activities 
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4-6 Computers/Classroom 


25+ Computers/Lab or 
Personal laptops for each learner 


easier 

• Makes it harder for instructors to 
ignore the technology; facilitates 
experimentation. 


• Lab Manager can provide 
instantaneous technical and 
instructional support 

• Can be implemented by networking 
only the lab 

• Power and cooling upgrades need 
only be done to one room 

• Security is higher (including 
avoidance of theft, damage, 
“hacking,” misuse) 

• Access before and after school hours 
may be possible 


Disadvantages 


Disadvantages 


• Prevents the entire class from 
engaging at one time in activities 
which require a 1:1 or 2:1 
learner:computer ratio (see Step 3) 

• Actual computer use may be limited 
to a few computer-literate learners, 
unless instructor encourages others 
to use the hardware 

• Instructors must be self-sufficient 
both technically and instructional^ 
in use of the technology 

• May require networking all 
classrooms 

• May require upgrades to power 
and/or cooling for all classrooms 

• Security more difficult 

• Access before and after school hours 
often not feasible 


• Usage must be planned and 
scheduled; access may be limited to 
scheduled times 

• Instructors may be tempted to 
“leave it to the lab manager” and not 
actively work with learners while in 
the lab 

• Instructors may not develop their 
own technology skills, but may 
“leave it to the lab manager” 

• Learners “pulled out” to go to the 
lab may be stigmatized 



How will we make the software available when and where 
it’s needed? 
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A common error is to purchase dozens, or even hundreds of CD-ROM’s 
with software which must be installed and used on individual computers. 
This rapidly creates an overwhelming logistical nightmare for instructors, 
learners and administrators: CD’s and learner records get lost, and they 
are never in the right place. For any but the smallest and simplest 
installations, delivery of software through local area networks (LANs) or 
the Internet is strongly recommended. The additional investment in 
networking, network management and instructor training is more than 
offset by the increased access to software, and its more efficient use. 

Use of a management system such as PLATO Pathways also is strongly 
recommended to ease access to software, as well as to maintain central 
records of use and achievement. This “just in time” network delivery 
allows you to maximize use of your software portfolio and prevent loss or 
damage. 

PLATO Pathways can create simple menus to launch any PLATO or non- 
PLATO Windows software, including access to specific Internet Web sites. 
PLATO LAN licensing limits only the number of simultaneous users, not 
the number of work stations or total users. While they are using PLATO 
software, learners can float among work stations on the network (or at 
home over the Internet) and still have access to assigned software while 
maintaining central data collection and administration. 

If your plans include the Internet, networking is all but mandatory. 
Individual learners and instructors working at home for distance 
education can cost-effectively use telephone lines and modems for Internet 
connections (though modestly priced broadband connections to homes are 
becoming available). However, as soon as simultaneous connections are 
needed by more than a handful of computers at a given site, a high-speed 
network connection will prove less expensive to operate, faster, and more 
reliable. Such connections are now common in schools. Most use 
telephone wire-like connections, but some are wireless (which makes it 
easier for learners to carry around laptop computers, or for teachers to 
rearrange classrooms). 

For best performance, reliability and security, add a central computer 
called a file server to your local area network, and use it to run the 
software you purchase. Some school districts network all their buildings 
together in a wide area network (WAN), and centralize the file servers for 
maximum efficiency of software use and network administration. 

If a LAN or a WAN is too complicated to be justified for your use, then you 
can use a network solely for Internet access. In this case, you can run 
PLATO on the Internet and still have private and secure access to learner 
records. 
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Step 7: Plan Technical Support 



Unfortunately, computers are neither as reliable nor as simple to use as 
other familiar technologies such as telephones and automobiles. 
Computers require knowledgeable setup and frequent attention to keep 
them running reliably. Failure to provide adequate and timely technical 
support is often cited in surveys of instructors as the greatest barrier to 
increased use of technology. 



The table below summarizes the typical levels of technical support 
required in an academic computer installation. 



Source of 
Technical 
Support 


Description 


Response 

Time 

Require- 

ment 


Instructor 


• Enrolling learners in groups 

• Locating and Assigning software 

• Generating reports on progress 

• Simple “troubleshooting"’ work station problems 
with mouse, locked-up computer, e-mail, etc. 


Immediate, 

ongoing 

On site 


Network 
Administrat 
or (may be 
same or 
different 
from 

instructor) 


• Enrolling classes 

• Exchanging data with central school 
administration system 

• Maintaining security settings 

• Routine backup copies of important data 


Daily, 
Ongoing 
On site 



ERjt 
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Source of 
Technical 
Support 


Description 


Response 

Time 

Require- 

ment 




• Installing new instructional software 

• Erasing old data and unused software 

• Simple troubleshooting of common network 
operational problems 

• Assists remote users (distance education 
learners) in resolving simple technical and use 
problems 




Network 
Manager 
(may be a 
contractor 
in small 
systems) 


• Designs and sets up network 

• Designs and sets up security system 

• Maintains network speed/reliability 

• Maintains speed/reliability of network-wide 
services such as Internet access and e-mail 

• Applies periodic software upgrades and 
maintenance 

• Troubleshoots network hardware and 
performance problems 


Weekly and 
on-demand, 
on site and 
via 

network 


Hardware 

and 

Software 

Suppliers 


• Provides regular software upgrades and 
maintenance 

• Consults with your technical support to resolve 
operational problems, incompatibilites, etc. 


On-demand 
via web 
site, e-mail 
and 

hotline; on- 
site 

consulting 
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Step 8: Plan Professional 
Development 



There is a growing concensus that a school technology budget should be 
allocated in the range of 1/3 to hardware, 1/3 to software, and 1/3 to 
professional development of instructors. 4 Adequate, timely and 
continuous professional development is absolutely key to effective 
technology implementation. 

In our experience, educators seem to progress through four levels of 
sophistication in the instructional use of technology. The levels are: 

1. Struggling with the technology . Simply learning the complexities of 
user interfaces, operating systems, how to use a given software system for 
productive work, and how to use the resources of the local network and/or 
the Internet is usually a major challenge. While this kind of learning is an 
ongoing process, mastering the basics is a requirement before any 
productive work in classrooms is possible. 

2. Supplementary Use . Once they’re past the basics of using a computer, 
it’s natural for instructors to approach the use of the new technology from 
the frame of reference of what they are already doing. This tends to lead 
instructors to use the technology to directly replace previous classroom 
activities on more or less a 1-for-l basis. Thus, instead of workbooks, 
learners work on computer drill-and-practice exercises. Instead of 
reviewing with the textbook; learners review with computer tutorials. 



4 For example, see: Panel on Educational Technology, Report to the President on the Use 
of Technology to Strengthen K-12 Education in the United States. Washington, DC: 
President’s Committee of Advisors on Science and Technology, March, 1997. 




Copyright ©TRO Learning, Inc. January 2000 All rights reserved. 
Tech Paper #5-Guide to Implementing Technology.doc 



31 



Page 29 



Instead of paper tests, learners work on computer tests. Instead of flash 
cards, learners work on computer memory exercises games. Instead of 
paper-and-pencil reports, learners use word processors and multimedia 
production tools, and create Web sites. Instead of researching the reports 
in the library, learners search for Web sites. 

3. Complementary Use . Sometimes, instructors go on to seek specific 
technology-based learning activities which are new to the curriculum, and 
which could be done by conventional means only with unacceptable cost, 
time or risk. Examples include use of simulations designed to teach 
problem-solving; use of computers for gathering and handling complex, 
“messy” data from real-world problems; use of the Internet e-mail, chat 
and multimedia capabilities to collaborate with classrooms and workers in 
other parts of the state, nation, or world. 

4. Primary Use. Some instructors take the technology further. They use 
it to fundamentally change their role, the structure of their classroom, or 
the structure of the school. These instructors often use technology to 
replace large-group instruction with highly individualized, self-paced 
study. They may completely restructure the curriculum into a problem- 
centered, interdisciplinary approach. They may exploit the distance 
education capabilities of the Internet to build community-based programs. 
In conventional secondary and post-secondary environments, these uses 
are most commonly found in learning labs for special needs 
(remediation/developmental studies, advanced work, etc.). With increased 
availability of technology, however, examples of these applications are 
growing in “mainstream” classrooms, as well. They are also found in a 
wide range of non-traditional settings, such as alternative schools, charter 
schools, home schools, workplace learning, lifelong learning, job training, 
etc. 

PLATO has components and features designed to be used in the top three 
levels. For example, PLATO problem-solving activities (PSA’s) are often 
used by instructors working at the complementary and primary levels. 
PLATO tutorial modules are often used by instructors working at the 
supplementary and primary levels, and they may play a supporting role at 
the complementary level. PLATO assessment capabilities are often used 
at all three levels (after the first). 

Appropriate professional development is critical to advancement from one 
level to the next. In our experience, without substantial training most 
instructors never progress beyond the first or second levels. Plan the 
training just before and in tandem with the introduction of new 
technologies, and should progress over a period of years as instructors 
develop experience with the technology. 



Types of Training 
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There are many types of software training available. Some of the training 
can best be done in workshops, while other types are best done in less 
formal settings. 

Workshops 

To plan workshops, you can use these “rules of thumb” to plan for 
professional development training your faculty and staff may need. These 
should be “hands on” experiences, so keep the learner:computer ratio at 
1:1, and limit class sizes to 10 per instructor. All of the training types 
below, except for the last two, are needed by instructors. 



Level and Type of Training 


Estimated Training 
Days per Person 


Level 1: Basic introduction to computers, Windows, 
Networks 


1-2 days per operating 
system 


Level 1: Productivity tools (such as word processors, 
spreadsheets, multimedia production tools, etc.) 


3-5 days per tool 


Level 1: Internet overview for educators 


2-3 days 


Level 2: Supplementary software and its use 


% to 3 days per software 
package, depending on 
size and complexity. 


Level 3: Complementary uses of software 


1-3 days per type 


Level 4: Primary uses of software 


3 days 

introduction/overview, 
plus V 2 to 5 days per 
curriculum, depending on 
size. 


Level 1: Administration/management of the 
system/network (for the system 

administrators/managers; not needed by instructors) 


Basic: 3 days 
Advanced: 2-4 days 


Level 1: Technical support of the system (for trained 
and certified computer professionals with network 
certification as needed) 


3-5 days per software 
system type 



Consultation 

Research on change management reinforces the importance and 
effectiveness of ongoing informal consultation as a follow-up to formal 
training experience. Instructors need to have easy access to expertise in 
both the technicalities of the software and the subtleties of educational 
use. 
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For PLATO, this role is handled by the Education Consultant (EC). We 
strongly recommend that you budget EC consultation time on a regular 
basis throughout the first 2-3 years of use of the system, and again 
whenever new software is introduced or new instructors start using it. 
Typical consultation budgets include 3-7 days per site in the first year, 
gradually declining to 1-3 days per site in the third year. 

You may also have access to experts within your own district or region. If 
so, be sure to arrange for their active involvement in your program. 



Where to Find Professional Development Training 

Sources of training include: 

• Expertise within your own staff 

• Local colleges 

• Local education support agencies or district specialists 

• National conferences and workshops 

• Major software vendors 

PLATO provides professional development services, including workshops 
and consultation for teachers and technical support personnel, through its 
large field organization of Education Consultants and Field Engineers. 
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Step 9: Plan Evaluation 



Feedback on how your instructors and learners are doing is a basic 
requirement for continuous improvement. It’s likely that you will want to 
know how well your technology investment is paying off. If there are 
problems, you’ll need to know quickly so you can intervene. And, it’s likely 
that other stakeholders who provided the funding for your technology 
investment will want you to demonstrate a return on the investment. The 
time to plan for the evaluation you need is at the beginning of the project. 

Evaluation models and guidelines are discussed extensively in PLATO’s 
Technical Paper #2: Guidelines for Evaluating PLATO. Be sure to obtain 
that paper as you plan your evaluation. 

Here are some answers to common questions on evaluation. 

What should we evaluate? 



The main purpose of your evaluation should be to measure progress 
toward achieving the goal you identified in Step 1, above. 

It is also often wise to evaluate how extensively and how well the 
technology plan was implemented. And, you may need to track program 
cost as well. 

Administrators often assume that an evaluation should compare classes 
using technology with those that don't. Unfortunately, that’s rarely 
practical. Presumably, the purpose of such a comparison is to judge the 
impact of technology. But to attribute results to the technology, you would 
have to take special precautions to be sure that the “experimental” and the 
’’control” classrooms differed in no other way. So, you would have to make 
sure they taught exactly the same content, in exactly the same way, with 
the same (or matched) instructors, randomly assigned learners, etc. 
Creating such an experiment often would be unacceptably disruptive and 
would have such an artificial situation that the results wouldn’t 
necessarily be applicable to your “normal” environment. Therefore, it’s 
usually best to concentrate the evaluation on how well the goal(s) you 
identified were achieved in your “real world” setting. 

When should we evaluate? 




At the beginning of the program, expectations are very high, but your 
organization has yet to climb the “learning curve” to successfully 
implement the technology. We often find that the best results occur in the 
second or later iterations of the curriculum. In academic settings, this 
usually means that evaluation for results should occur in the second 
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semester or year of the program. Evaluations before then should monitor 
success of the implementation compared to the plan. Only when the 
implementation is going smoothly and according to plan should evaluation 
focus primarily on results. 

What data should we gather? 

The data you gather depends on the goal of the evaluation. Here are some 
commonly-used sources of data: 

• Pre-tests: a test administered just before the program starts should 
verify the entering skill levels of the learners and their readiness to 
learn what is planned. A placement test usually screens for 
prerequisite skills and will allow you to verify readiness of the 
learners. A parallel form of the post-test (see below) will allow you to 
verify that your learners didn't already know what is to be taught, but 
may not be detailed enough to verify mastery of prerequisites. 

• Post-tests: your program may require an exit test such as a state- 
mandated competency test or a standardized test. Alternatively, you 
may wish to use the PLATO Custom Assessment Test tool to construct 
a “final exam" from PLATO questions. Or, you may choose to write 
your own test. 

Depending on your goal, the post-test results may be an adequate 
measure of how well you achieved it. However, if the goal involves a 
performance improvement (rather than a knowledge gain), then a post- 
test can serve only as a measure of program implementation, not as 
your ultimate “criterion” measure. 

• Performance measures: if your program's goals focus on 
performance, then you will need to measure that performance. 
Examples include reduction of dropout rate, employment rate, 
reduction of error rate, or improvement of a productivity measure. 

• Demographics: it may be important to track specific subgroups of 
learners, or to otherwise characterize the profile of the learners in 
ways that are significant to you or to other stakeholders. If so, be sure 
to figure out the identifiers before learners enroll, so you can gather 
the data and enter it into the PLATO Pathways management system, to 
use it to generate reports specific to those groups. Common 
demographic data include age, years in school, ethnicity, free/reduced 
lunch, program type, referring institution or instructor, primary 
instructor, total years of education or experience, etc. 

Additional measures are often useful for monitoring successful 
implementation. Some common ones include: 

• Number of modules tried and mastered: PLATO Pathways tracks 
the number of modules a learner enters, how often they have tried a 
given lesson, and whether the learner passed the module mastery test. 

• Time on task: PLATO Pathways tracks total time spent on each 
activity, as well as when the activity occurred. 



0 
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• Subjective reaction of learners: research has shown consistently 
that a learner's subjective reaction to a learning experience is virtually 
unrelated to how much they learn from it. However, overall 
satisfaction with the learning experience may be a useful indicator of 
program implementation success. It may also provide some suggestion 
of how likely learners are to persist in the program. 

• Subjective reaction of instructors: instructors’ reactions are 
strongly influenced by a wide range of factors, such as ease of use, level 
of their training, availability of technical support, etc., as well as 
instructional effectiveness. Therefore, assessment of subjective 
reactions can be a useful indicator of program implementation success. 

Note that sample learner and instructor questionnaires are included in 

Technical Paper #2. 

• Cost: you may need to monitor program costs, both for initial set-up 
and for ongoing operation of the program. Issues involved in cost 
modeling and cost comparison are discussed in Technical Paper # 2 . 
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Conclusion 



The nine planning tasks described in this paper are based on our 
understanding of the factors which are critical to the success of most 
educational applications of technology, including PLATO. Our intent is to 
help guide you toward a successful technology implementation. We 
recognize that PLATO is often only one part of a much larger technology 
plan, and we have designed the system to help you in this kind of use. 
Similarly, in this paper, we have attempted to discuss the issues generally 
enough so you can apply them to your entire technology plan, not just the 
PLATO portion. 

For small implementations, these eight planning tasks can be very simple 
and need not consume much effort. Often, in such cases, we find that they 
can efficiently be discussed in a single implementation meeting of an hour 
or two. For large implementations, of course, the planning process is much 
more complex and may require months of work with many individuals. 

Regardless of program size, however, in our experience it is important to 
address all nine of the planning tasks identified here. 

PLATO’s professionals are trained and committed to support your 
planning process at each step. Consult with your Account Manager and 
Education Consultant for advice, at any time before or after your decision 
to purchase PLATO. Before you buy, your Account Manager can help you 
make some of the key decisions described here, in order to be sure you are 
buying what you need to achieve your goals. After your purchase, your 
Education Consultant will help will the remaining questions during 
implementation planning. Both the AM and EC have at their disposal the 
full resources of technical and educational specialists at TRO Learning, 
Inc., and can call on them to help you at any time. 
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